Search

Abstract Feature matching plays a crucial role in computer vision, with applications in visual localization, simultaneous localization and mapping (SLAM), image stitching, and more. It establishes correspondences between sets of feature points from multiple images, enabling various tasks. Over the years, feature matching has witnessed significant development, with an increasing number of methods being applied. However, different methods exhibit different degrees of applicability in different scenarios and requirements due to their different rationales. To cope with these issues, a comprehensive analysis and comparison of matching methods are essential. Existing reviews often lack coverage of deep learning models and focus more on feature detection and description, neglecting the matching process. This survey investigates feature detection, description, and matching techniques within the feature‐based image‐matching pipeline. Representative methods, their mechanisms, and application scenarios are also briefly introduced. In addition, comprehensive evaluations of classical and state‐of‐the‐art methods are conducted through extensive experiments on representative datasets. Particularly, matching‐based applications are compared to fully demonstrate the advantages of the methods. Lastly, this survey highlights current problems and development directions in matching methods, serving as a reference for researchers in the field.

Abstract Identifying saliency in high dynamic range (HDR) images is a fundamentally important issue in HDR imaging, and plays critical roles towards comprehensive scene understanding. Most of existing studies leverage hand‐crafted features for HDR image saliency prediction, lacking the capabilities of fully exploiting the characteristics of HDR image (i.e. wider luminance range and richer colour gamut). Here, systematical studies are carried out on HDR image saliency prediction by proposing a new framework to single out the contributions from multi‐exposure images. Specifically, inspired by the mechanism of HDR imaging, the method first utilizes graph neural networks to model the relations among multi‐exposure images and the tone‐mapped image obtained from an HDR image, enabling more discriminative saliency‐related feature representations. Subsequently, the saliency features driven by global semantic knowledge are aggregated from the tone‐mapped image through enhancing global context‐aware semantic information. Finally, a fusion module is designed to integrate saliency‐oriented feature representations originated from multi‐exposure images and the tone‐mapped image, producing the saliency maps of HDR images. Moreover, a new challenging HDR eye fixation database (HDR‐EYEFix) is created, expecting to further contribute the research on HDR image saliency prediction. Experiment results show that the method obtains superior performance compared to the state‐of‐the‐art methods.

Abstract The acquisition cycle of remote sensing images is slow, and the labelling process encounters challenges, which have become prominent with the rapid development of remote sensing image object detection research. Therefore, this article provides a way to make the model better capture the diversity and contextual relationships in the data, and solve this problem by more than just data augmentation. Specifically, this method is a few‐shot object detection method for remote sensing based on global context combined with implicit knowledge decoupled head (GC‐IKDH). This method first uses a segmentation strategy to convert high‐resolution images into low‐resolution images and expands the sample size through a generative model. Secondly, GC attention is introduced to generate a GC vector by weighting and averaging the information of each position in the input sequence, which helps the model better understand the semantics of the input sequence. Finally, an IKDH is added to improve the model head, which is used to learn specific features in the data so that the model can better handle the diversity in the data. Experimental results show that GC attention and IKDH boosting provide a good performance boost to the baseline model. Compared with other few‐shot samples, this method achieves state‐of‐the‐art performance under different shot settings and highly competitive results on two benchmark datasets (NWPU VHR‐10 and DIOR).

Abstract Most existing multi‐view stereo (MVS) methods fail to consider global context information in the stage of feature extraction and cost aggregation. As transformers have shown remarkable performance on various vision tasks due to their ability to perceive global contextual information, this paper proposes a transformer‐based feature enhancement network (TF‐MVSNet) to facilitate feature representation learning by combining local features (both 2D and 3D) with long‐range contextual information. To reduce memory consumption of feature matching, the cross‐attention mechanism is leveraged to efficiently construct 3D cost volumes under the epipolar constraint. Additionally, a colour‐guided network is designed to refine depth maps at a coarse stage, hence reducing incorrect depth predictions at a fine stage. Extensive experiments were performed on the DTU dataset and Tanks and Temples (T&T) benchmark and results are reported.

Abstract Existing segmentation‐based methods have made considerable progress in arbitrarily shaped text detection due to the advantage of dealing with shape variation. However, there still exist challenges to detecting accurate text instances with dense layouts, inaccurate annotations, and complex backgrounds. Many recent works have focused on improving arbitrary boundary prediction, but it may be difficult to accurately distinguish each instance of dense layouts because their boundary pixels may be mistakenly classified to produce inaccurate results (i.e., adhesive texts) with inaccurate annotation and complex backgrounds. Considering the local and long‐range dependencies, this paper proposes an efficient text detector, namely T‐Skeleton, to obtain more reliable segmentation detections. In the spirit of object skeletonization, we introduce the text instance skeleton highlighting the semantically significant structure (similar to the skeleton of a fish) to explicitly capture the long‐range dependencies of text instances. The key idea of T‐Skeleton is to calibrate the coarse text proposals by embedding text instance skeletons to separate crowd texts accurately and robustly. We further design a channel attention module to enlarge the performance margin between T‐Skeleton and the segmentation baseline. Experimental results on four publicly available datasets show the superiority of T‐Skeleton in handling long and curved texts.

Abstract With the popularity of handheld devices, video stabilization is becoming increasingly important. In previous studies, many methods have been proposed to stabilize shaky videos. However, these methods fail to balance between image content integrity and stability. Some methods sacrifice image content for better stability. Other methods ignore the subtle jitters, which leads to poor stability. This work innovatively proposes a video stabilization method based on decomposed motion compensation. First, a grid‐based motion statistics method is adopted for motion estimation, which obtains more accurate motion vectors according to matched likelihood estimates. Then, the motion compensation is inherently decomposed into two parts: linear motion compensation and auxiliary motion compensation. Linear motion compensation removes complex jitter by constructing linear path constraints to obtain a more stable camera path. Auxiliary motion compensation uses a moving average filter to remove the high‐frequency jitter as a supplement and preserve more image content. The two components are combined with individual weights to derive the final transform matrix and warp the original frames. Experimental results show that our method outperforms the previous methods on NUS and DeepStab datasets qualitatively and quantitatively.

Abstract Pixel‐level labeling of regions of interest in an image is a key step in building a labeled training dataset for supervised deep learning networks of images. However, traditional manual labeling of cancerous regions in digital pathological images by doctors is time‐consuming and inefficient. To address this issue, this paper proposes an automatic labeling method for whole slide images, which combines clustering and edge detection techniques. The proposed method utilizes the multi‐level feature fusion model and the Long‐Short Term Memory network to discriminate the cancerous nature of the whole slide images, thereby improving the classification accuracy of the whole slide images. Subsequently, the automatic labeling of cancerous regions is achieved by integrating a density‐based clustering algorithm and an edge point extraction algorithm, both based on the discriminated results of the cancerous properties of whole slide images. The experimental results demonstrate the effectiveness of the proposed method, which offers an efficient and accurate solution to the challenging task of cancerous region labeling in digital pathological images.

Abstract Tumour medical image segmentation plays a crucial role in clinical imaging diagnosis. Existing research has achieved good results, enabling the segmentation of three tumour regions in MRI brain tumour images. Existing models have limited focus on the brain tumour areas, and the long‐term dependency of features is weakened as the network depth increases, resulting in blurred edge segmentation of the targets. Additionally, considering the excellent segmentation performance of the Swin Transformer(Swin‐T) network, its network structure and parameters are relatively large. To address these limitations, this paper proposes a brain tumour segmentation framework with deep nuanced reasoning and Swin‐T. It is mainly composed of the backbone hybrid network (BHN) and the deep micro texture extraction module (DMTE). The BHN combines the Swin‐T stage with a new downsampling transition module called dual path feature reasoning (DPFR). The entire network framework is designed to extract global and local features from multi‐modal data, enabling it to capture and analyze deep texture features in multi‐modal images. It provides significant optimization over the Swin‐T network structure. Experimental results on the BraTS dataset demonstrate that the proposed method outperforms other state‐of‐the‐art models in terms of segmentation performance. The corresponding source codes are available at https://github.com/CurbUni/Brain‐Tumor‐Segmentation‐Framework‐with‐Deep‐Nuanced‐Reasoning‐and‐Swin‐T.

Abstract The Intelligent Evaluation System for Calligraphy Characters (IESCC) is used for teaching calligraphy, and users can learn calligraphy through the modifications given by the system. Chinese character skeleton extraction is an important step in the intelligent evaluation algorithm of calligraphic characters. The skeletons of Chinese characters extracted by traditional refinement algorithms are prone to redundant branches and deformed skeletons, which can lead to skeleton extraction results that do not conform to the topology of the original character. In this study, the focus lies on hard‐pen regular script, and skeleton repair and extraction are performed for these characters. According to the writing characteristics of regular script, the redundant burs are removed and the deformation zone of the thinned skeleton is detected, and then the idea of first splitting is used, then restructuring, to propose a skeleton extraction algorithm based on stroke characterization and ambiguous zone detection for hard‐pen regular script, referred to as SCAD. First, a thinning algorithm is used to extract the skeleton of Chinese characters and remove redundant pixels. By analyzing the stroke characteristics of regular script, the burrs are classified and different conditions are set to detect and remove the burrs. Then the ambiguous zones are detected according to the different kinds of junction points. Then, curvature, stroke width and direction deviation are used to analyze the continuity of stroke segments, and the decision function is used to classify the stroke segments. Finally, the stroke segments with optimal pairings were compensated by interpolation according to the direction trend. This concludes the skeleton extraction. Skeleton extraction is performed on 1000 sample characters, and the SCAD algorithm can extract the skeleton of Chinese characters with an accuracy of up to 98.37%. It is proved that the SCAD method proposed here is a practical and effective method to extract the skeleton of hard‐pen regular script.

Abstract The limited of texture details information in low‐resolution facial or eye images presents a challenge for gaze estimation. To address this, FSKT‐GE (feature maps similarity knowledge transfer for low‐resolution gaze estimation) is proposed, a gaze estimation framework consisting of both a high resolution (HR) network and low resolution (LR) network with the identical structure. Rather than mere feature imitation, this issue is addressed by assessing the cosine similarity of feature layers, emphasizing the distribution similarity between the HR and LR networks. This enables the LR network to acquire richer knowledge. This framework utilizes a combination loss function, incorporating cosine similarity measurement, soft loss based on probability distribution difference and gaze direction output, along with a hard loss from the LR network output layer. This approach on low‐resolution datasets derived from Gaze360 and RT‐Gene datasets is validated, demonstrating excellent performance in low‐resolution gaze estimation. Evaluations on low‐resolution images obtained through 2×, 4×, and 8× down‐sampling are conducted on two datasets. On the Gaze360 dataset, the lowest mean angular errors of 10.97°, 11.22°, and 13.61° were achieved, while on the RT‐Gene dataset, the lowest mean angular errors of 6.73°, 6.83°, and 7.75° were obtained.

Abstract In the field of scientific visualization, the upscaling of time‐varying volume is meaningful. It can be used in in situ visualization to help scientists overcome the limitations of I/O speed and storage capacity when analysing and visualizing large‐scale, time‐varying simulation data. This paper proposes self‐attention residual network‐based spatial super‐resolution (SARN‐SSR), a spatial super‐resolution model based on self‐attention residual networks that can generate time‐varying data with temporal coherence. SARN‐SSR consists of two components: a generator and a discriminator. The generator takes the low‐resolution volume sequences as the input and gives the corresponding high‐resolution volume sequences as the output. The discriminator takes both synthesized and real high‐resolution volume sequence as the input and gives a matrix to predict the realness as the output. To verify the validity of SARN‐SSR, four sets of time‐varying volume datasets are applied from scientific simulation. In addition, SARN‐SSR is compared on these datasets, both qualitatively and quantitatively, with two deep learning‐based techniques and one traditional technique. The experimental results show that by using this method, the closest time‐varying data to the ground truth can be obtained.

Abstract Scene text in natural images carries additional semantic information to aid in image classification. Existing methods lack full consideration of the deep understanding of the text and the visual text relationship, which results in the difficult to judge the semantic accuracy and the relevance of the visual text. This paper proposes image classification based on Cross modal Knowledge Learning of Scene Text (CKLST) method. CKLST consists of three stages: cross‐modal scene text recognition, text semantic enhancement, and visual‐text feature alignment. In the first stage, multi‐attention is used to extract features layer by layer, and a self‐mask‐based iterative correction strategy is utilized to improve the scene text recognition accuracy. In the second stage, knowledge features are extracted using external knowledge and are fused with text features to enhance text semantic information. In the third stage, CKLST realizes visual‐text feature alignment across attention mechanisms with a similarity matrix, thus the correlation between images and text can be captured to improve the accuracy of the image classification tasks. On Con‐Text dataset, Crowd Activity dataset, Drink Bottle dataset, and Synth Text dataset, CKLST can perform significantly better than other baselines on fine‐grained image classification, with improvements of 3.54%, 5.37%, 3.28%, and 2.81% over the best baseline in mAP, respectively.

Abstract This paper proposes a super‐resolution reconstruction model, SRPGANto improve the visual quality of images based on generative adversarial networks (GANs) by improving the network structures of the generator and the discriminator. In the generator, a dual branch residual block is designed instead of the residual block, including a branch with an attention mechanism and a branch without an attention mechanism, to extract more differentiated features. Normalization methods are explored to avoid unstable training and bath normalization artifacts and use a half instance normalization layer that is more suitable for underlying visual problems compared with traditional batch normalization. In the discriminator, PatchGAN is applied instead of typical GAN to improve the generation of local texture by discriminating each patch rather than the global image. The experimental results on the public datasets demonstrate that the proposed SRPGAN can achieve excellent quantitative evaluation while improving the visual quality of reconstructed images.

Abstract Despite the existence of various super‐resolution (SR) methods, most of them focus on designing models for specific upscaling factors rather than fully exploiting inter‐scale correlation to improve efficiency. In contrast, multi‐scale SR methods can effectively reduce the redundancy of network parameters by aggregating the feature extraction processes corresponding to multiple scales into a unified process. The aim of this study is to enhance the compactness and efficiency of the SR model. Thus, an efficient multi‐scale SR method called the diverse branch feature refinement network (DBFRN) is proposed. By decoupling the training process and inference process based on the idea of structural re‐parameterization, multi‐branch topology is adopted to enrich multi‐scale learning and merge branches to achieve efficient inference with equivalent effects. Specifically, two re‐parameterization strategies are designed and two corresponding feature refinement blocks for different feature levels in multi‐scale SR network. Extensive experiments demonstrate that the proposed multi‐scale SR method is effective and efficient, and it can outperform advanced single‐scale methods in terms of quantity and quality.

Abstract Whole brain segmentation from magnetic resonance images (MRI) is crucial in diagnosing brain diseases and analyzing neuroimaging data. Despite advances through deep learning, challenges such as uneven gray distribution and the presence of artifacts still present hurdles in medical image processing. These limitations are often a result of insufficient spatial contextual information and lack of attention to important regions within existing models. To address these issues, this paper presents SSA‐UNet (Squeeze‐and‐Excitation and Self‐Attention UNet), a uniquely designed deep convolutional neural network that integrates spatial constraints by converting three consecutive 2D MRI slices into a single 2.5D image. This facilitates capturing inter‐slice dependencies effectively. Additionally, the newly formulated SSA block, which sequentially incorporates channel attention and Self‐Attention mechanisms, is placed before the decoders in the conventional U‐Net architecture. This enables the network to automatically weight different feature maps and focus more effectively on regions requiring precise segmentation. Rigorous evaluations on LPBA40 and IBSR18 datasets substantiate the remarkable improvements in accuracy and stability achieved by SSA‐UNet. Results indicate Dice coefficients of 98.38% and 97.47%, specificity of 99.69% and 99.57%, and sensitivity of 98.5% and 97.98% for the respective datasets. Compared to other existing models, SSA‐UNet shows significant improvements on both the LPBA40 and IBSR18 datasets. On the LPBA40 dataset, SSA‐UNet's Dice coefficient improved by 0.33% compared to the sub‐optimal model, while on the IBSR18 dataset, the improvement reached 1.78%. These empirical findings demonstrate SSA‐UNet's heightened capability in addressing the long‐standing challenges in MRI‐based whole‐brain segmentation.

Abstract The synthesis of rain fields is essential in multiple research fields and applications, including Single‐image Derain. However, there is a lack of research on simulated rain fields, and the existing rain field generation models struggle to capture complex spatial distributions and generate truly random rain fields. To address this, the authors propose a generative adversarial networks‐based rain field generation network, which consists of a generator, a discriminator, and a feature extraction block that can produce realistic and complex rain fields. The authors’ experiments demonstrate that this method achieves an average Frechet Inception Distance score of 0.035, and user studies indicate that the generated rain distribution looks naturally.

Abstract Learning the full extent of pixel‐level instance response in a weakly supervised manner remains unsatisfactory. Peak response maps (PRMs) localizes the discriminative object regions but cannot provide complete instance information, suffering from incomplete segmentation and unreliable mask prediction by noisy proposal retrieval. This work tackles this challenging problem by mining diverse class peak responses that include more discriminative and complete object regions and retrieving more reliable proposals from noisy segment proposal galleries. First, the existing method is enhanced with two more classification branches, thus contributing to more diverse and abundant instance regions from peak response maps. The mined class peak responses from two of the branches are then merged to generate more complete peak response maps by a clustering approach in their deep feature space. Then, instance segmentation masks are retrieved from a noisy object segment proposal gallery with class confidence, which is calculated by a normal classifier to obtain cleaner mask prediction. Finally, the pseudo‐supervision can be used to train an instance segmentation network in a fully supervised manner. Experiments on the PASCAL VOC 2012 dataset and COCO dataset show that the approach works effectively and outperforms other counterparts by a margin of more than 6 %, 4%, and 3% with the mean average precision (mAP) at IoU threshold of 0.25, 0.5 and 0.75, respectively.

Abstract Infrared object detection and tracking in dense urban traffic remain a challenge due to factors such as low contrast, small intra‐class differences, and frequent false positives and negatives. To overcome these, the authors introduce YOLO‐IR, an algorithm based on the enhanced YOLOv8s, and YOLO‐DeepOC‐IR, a comprehensive infrared multi‐object tracking method for urban traffic, integrating both detection and tracking. During preprocessing, three infrared image enhancement techniques, local contrast multi‐scale enhancement, non‐local means, and contrast limited adaptive histogram equalization, are applied for better reliability in dense scenes. To further improve the performance, the original YOLOv8s backbone is replaced with MobileVITv3 to enhance detection accuracy and robustness. This infrared feature extraction module, incorporated into the detector, combines canny edge detection, Gabor filtering, and open operation layers, significantly boosting object detection in infrared imagery. The tracker's feature processing capabilities are improved using the learned arrangements of three patch codes descriptor and locality‐sensitive hashing for feature extraction and matching. Experimental results on FLIR ADAS v2 and InfiRay datasets indicate superior performance of this method, achieving 78.6% mAP and 151.1 FPS in detection, and up to 80.8% moving object tracking accuracy, 78.6% identification F1 score, and 62.1% higher order tracking accuracy in multi‐object tracking.

Abstract Few‐shot learning enables machines to learn efficiently from limited labelled data. However, existing few‐shot learning methods may perform poorly when there is a lack of sufficient samples, and may encounter problems such as domain shift or overfitting when applied to new domains or tasks. To address the issues of poor fitting and insufficient generalization ability in new domains, a new method called triplet merged network with involution operators (TMNIO) is proposed. This method employs dual encoders that extract common and distinctive features from the prototype network, thereby enhancing the model's feature extraction capability. To further improve this ability, the traditional convolutional kernels are replaced with involution operators, which not only reduce the parameter count but also enlarge the receptive field to better extract local feature information. Additionally, this method employs a two‐stage training strategy, where triplet loss is used in the first stage to train the model and enhance its robustness and generalization ability. Extensive experiments on the miniImageNet, Omniglot, and Caltech‐UCSD Birds‐200 (CUB) datasets have shown that our proposed method achieved significant improvements in both training speed and accuracy, particularly on the miniImageNet dataset, where it achieved an outstanding 10% performance improvement.

Abstract Object detection in remote sensing images poses great challenges due to the dense distribution, arbitrary orientation, and aspect ratio variations of objects. Most of the existing methods rely on aligned convolutional features, which fail to capture the geometric information of objects effectively and result in the inconsistency between the classification score and localization accuracy. Moreover, densely packed objects suffer from spatial feature aliasing caused by the intersection of reception fields between objects. To address this issue, a deformable convolution‐based method named rotated points is proposed, which consists of two modules: a point set loss module and a high‐quality sample assignment module. The point set loss module can extract geometric features of objects in arbitrary directions with fine‐grained point sets for feature representation and introduce outlier penalties to penalize outlier points. The high‐quality sample assignment module measures the classification and localization ability, orientation quality, and point‐wise correlation of point sets comprehensively to enhance the consistency of classification and regression significantly. Experiments on the DOTA and FAIR1M datasets demonstrate that the proposed method achieves significant improvements over the benchmark model.

Abstract With recent advancements in robotic surgery, notable strides have been made in visual question answering (VQA). Existing VQA systems typically generate textual answers to questions but fail to indicate the location of the relevant content within the image. This limitation restricts the interpretative capacity of the VQA models and their ability to explore specific image regions. To address this issue, this study proposes a grounded VQA model for robotic surgery, capable of localizing a specific region during answer prediction. Drawing inspiration from prompt learning in language models, a dual-modality prompt model was developed to enhance precise multimodal information interactions. Specifically, two complementary prompters were introduced to effectively integrate visual and textual prompts into the encoding process of the model. A visual complementary prompter merges visual prompt knowledge with visual information features to guide accurate localization. The textual complementary prompter aligns visual information with textual prompt knowledge and textual information, guiding textual information towards a more accurate inference of the answer. Additionally, a multiple iterative fusion strategy was adopted for comprehensive answer reasoning, to ensure high-quality generation of textual and grounded answers. The experimental results validate the effectiveness of the model, demonstrating its superiority over existing methods on the EndoVis-18 and EndoVis-17 datasets.

Abstract Green hydrogen production is crucial for a sustainable future, but current catalysts for the oxygen evolution reaction (OER) suffer from slow kinetics, despite many efforts to produce optimal designs, particularly through the calculation of descriptors for activity. In this study, we develop a dataset of density functional theory calculations of bulk and surface perovskite oxides, and adsorption energies of OER intermediates, which includes compositions up to quaternary and facets up to (555). We demonstrate that per-site properties of perovskite oxides such as Bader charge or band center can be tuned through element substitution and faceting, and develop a machine learning model that accurately predicts these properties directly from the local chemical environment. We leverage these per-site properties to identify promising perovskites with high theoretical OER activity. The identified design principles and promising materials provide a roadmap for closing the gap between current artificial catalysts and biological enzymes such as photosystem II.

Abstract Piezoelectricity of organic polymers has attracted increasing interest because of several advantages they exhibit over traditional inorganic ceramics. While most organic piezoelectrics rely on the presence of intrinsic local dipoles, a highly nonlocal electronic polarisation can be foreseen in conjugated polymers, characterised by delocalised and highly responsive π-electrons. These 1D systems represent a physical realisation of a Thouless pump, a mechanism of adiabatic charge transport of a topological nature which results, as shown in this work, in anomalously large dynamical effective charges, inversely proportional to the bandgap energy. A structural (ferroelectric) phase transition further contributes to an enhancement of the piezoelectric response reminiscent of that observed in piezoelectric perovskites close to morphotropic phase boundaries. First-principles density functional theory (DFT) calculations performed in two representative conjugated polymers using hybrid functionals, show that state-of-the-art organic piezoelectric are outperformed by piezoelectric conjugated polymers, mostly thanks to strongly anomalous effective charges of carbon, larger than 5e—ordinary values being of the order of 1e—and reaching the giant value of 30e for band gaps of the order of 1 eV.

Abstract Despite the explosion of interest in topological materials over the last decades, their applications remain limited due to challenges in growth and incorporation with today’s microelectronics. As a potential bridge to close this gap, we investigate the group-IV alloy Ge 1–x Pb x , in the Ge-rich condition using density functional theory and show that relatively low concentrations of Pb (~9.4%) can lead to a topological phase transition. Furthermore, the calculation of the Z2 invariant for both the random alloy and the alloy with short-range order (SRO) indicate that the topological phase of the material can be directly modified by the degree of SRO. These findings are understood in terms of local structural relaxation, which decreases the bandgap in the random alloy. However, in the SRO case, the mutual avoidance of Pb leads to minimal structural relaxation, alleviating strain. Our findings not only highlight the emerging importance of SRO in alloy properties but also indicate the possibility of constructing topological interfaces between materials of identical composition (and nominally identical structure). Moreover, they uncover a viable avenue toward the monolithic integration of quantum materials with today’s semiconductor industry.

Abstract Cold metals with an energy gap around the Fermi level have been shown a great potential for reducing the power dissipation of transistors and diodes. However, only a limited number of 2D cold metals have been studied. In this work, we explored 3D cold metals through a systematic material search and found 252 types in the database. We performed first-principles calculations to investigate the conductance and work functions of 30 cold metals for material selection. Additionally, we studied the thermoelectric properties of four typical cold metals, which possess much larger Seebeck coefficients and figure-of-merits than conventional metals, by one and two orders of magnitude, respectively. Specifically, we constructed a monolayer MoS2 transistor with a cold metal contact of ZrRuSb. Our quantum transport simulations indicate that cold metal contacted MoS2 FETs exhibit a subthreshold swing smaller than 60 mV decade−1 over four decades, and on-state currents over 1 mA μm−1 are achieved at a supply voltage of 0.5 V. Our research provides a theoretical foundation and material basis for exploring 3D cold metals in developing electronic and thermoelectric devices.

Abstract The interplay of polarization and magnetism in materials with light can create rich nonlinear magneto-optical (NLMO) effects, and the recent discovery of two-dimensional (2D) van der Waals magnets provides remarkable control over NLMO effects due to their superb tunability. Here, based on first-principles calculations, we reported giant NLMO effects in CrI3-based 2D magnets, including a dramatic change of second-harmonics generation (SHG) polarization direction (90°) and intensity (on/off switch) under magnetization reversal and a 100% SHG circular dichroism effect. We further revealed that these effects could not only be used to design ultra-thin multifunctional optical devices but also to detect subtle magnetic orderings. Remarkably, we analytically derived conditions to achieve giant NLMO effects and proposed general strategies to realize them in 2D magnets. Our work not only uncovers a series of intriguing NLMO phenomena but also paves the way for both fundamental research and device applications of ultra-thin NLMO materials.

Abstract The Internet of Things (IoT) devices are pervasively deployed and embedded into our daily lives. Over several years, the massive assimilation of IoT devices has given rise to smart cities, smart factories, smart farms, etc. While IoTs can collect immense amounts of user and environmental information, the lack of adequate security measures is among the major concerns about information security and privacy. With the ever-increasing presence of IoTs, information flow between IoT devices in a single or multi-layered network is becoming increasingly vulnerable to attacks. Although several software-based solutions have been developed over the years that improve the security of resource-constrained IoT devices, there are limited efforts on hardware-oriented IoT security solutions. This paper proposes a Trusted HArdware-based Scalable Secure Architecture (THASSA) for IoT Edge systems. Leveraging standard cryptographic algorithms, THASSA provides a secure mechanism for data transfer and off-loads cryptography operations to trusted hardware. The modularized THASSA architecture allows devices to be added or removed without user intervention. Also, THASSA could be scaled to multiple subsystems using multiple trusted hardware to form IoT clusters with secure data path(s) for mission-critical applications. An experimental study has verified the correctness and effectiveness of THASSA.

Abstract This study addresses a limitation of prior research on pectoralis major (PMaj) thickness changes during the pectoralis fly exercise using a wearable ultrasound imaging setup. Although previous studies used manual measurement and subjective evaluation, it is important to acknowledge the subsequent limitations of automating widespread applications. We then employed a deep learning model for image segmentation and automated measurement to solve the problem and study the additional quantitative supplementary information that could be provided. Our results revealed increased PMaj thickness changes in the coronal plane within the probe detection region when real-time ultrasound imaging (RUSI) visual biofeedback was incorporated, regardless of load intensity (50% or 80% of one-repetition maximum). Additionally, participants showed uniform thickness changes in the PMaj in response to enhanced RUSI biofeedback. Notably, the differences in PMaj thickness changes between load intensities were reduced by RUSI biofeedback, suggesting altered muscle activation strategies. We identified the optimal measurement location for the maximal PMaj thickness close to the rib end and emphasized the lightweight applicability of our model for fitness training and muscle assessment. Further studies can refine load intensities, investigate diverse parameters, and employ different network models to enhance accuracy. This study contributes to our understanding of the effects of muscle physiology and exercise training.

Abstract Polymers play an integral role in various applications, from everyday use to advanced technologies. In the era of machine learning (ML), polymer informatics has become a vital field for efficiently designing and developing polymeric materials. However, the focus of polymer informatics has predominantly centered on single-component polymers, leaving the vast chemical space of polymer blends relatively unexplored. This study employs a high-throughput molecular dynamics (MD) simulation combined with active learning (AL) to uncover polymer blends with enhanced thermal conductivity (TC) compared to the constituent single-component polymers. Initially, the TC of about 600 amorphous single-component polymers and 200 amorphous polymer blends with varying blending ratios are determined through MD simulations. The optimal representation method for polymer blends is identified, which involves a weighted sum approach that extends existing polymer representation from single-component polymers to polymer blends. An AL framework, combining MD simulation and ML, is employed to explore the TC of approximately 550,000 unlabeled polymer blends. The AL framework proves highly effective in accelerating the discovery of high-performance polymer blends for thermal transport. Additionally, we delve into the relationship between TC, radius of gyration (R g), and hydrogen bonding, highlighting the roles of inter- and intra-chain interactions in thermal transport in amorphous polymer blends. A significant positive association between TC and R g improvement and an indirect contribution from H-bond interaction to TC enhancement are revealed through a log-linear model and an odds ratio calculation, emphasizing the impact of increasing R g and H-bond interactions on enhancing polymer blend TC.

Abstract The research of new electrode materials such as sodium intercalation compounds is key to meet the challenges of future demands of sustainable energy storage. For these batteries, the intercalation behavior on the micro-scale is governed by a complex interplay of chemical, electrical and mechanical forces strongly influencing the overall cell performance. The multiphase-field method is a suitable tool to study these multi-physics and bridge the scale from ab-initio methods to the cell level. In this work, we follow a combined approach of experiments, density functional theory (DFT) calculations and multiphase-field simulations to predict thermodynamic and kinetic properties for the P2-type Na X Ni1/3Mn2/3O2 sodium-ion cathode material. Experimentally, we obtain the thermodynamic potential and diffusion coefficients at various sodium contents using electrochemical techniques and discuss limitations of the experimentally applied methods. DFT is used to identify stable phases by calculating an energy hull curve. Then, the influence of long-range dispersion interactions and the exchange-correlation functional on the voltage curve is investigated by comparison with experimental results. Finally, multiphase-field simulations are performed based on inputs from experiments and DFT. The fitting of phase-specific chemical free energies from DFT calculations and experimental data is discussed. Our results highlight the thermodynamic consistency of all three approaches close to thermodynamic equilibrium. Furthermore, the phase-field method accurately describes the kinetics of the system including multiple phase transitions, by which we unravel the mechanism of the P2-O2 phase transition in a single crystal under the influence of intercalation reaction, bulk diffusion and elastic deformation. The model is able to predict the kinetic capacity loss depending on charging rate in agreement with C-rate experiments.

Abstract Two-dimensional (2D) materials combine many fascinating properties that make them more interesting than their three-dimensional counterparts for a variety of applications. For example, 2D materials exhibit stronger electron-phonon and electron-hole interactions, and their energy gaps and effective carrier masses can be easily tuned. Surprisingly, published band gaps of several 2D materials obtained with the G W approach, the state-of-the-art in electronic-structure calculations, are quite scattered. The details of these calculations, such as the underlying geometry, the starting point, the inclusion of spin-orbit coupling, and the treatment of the Coulomb potential can critically determine how accurate the results are. Taking monolayer MoS2 as a representative material, we employ the linearized augmented planewave + local orbital method to systematically investigate how all these aspects affect the quality of G 0 W 0 calculations, and also provide a summary of literature data. We conclude that the best overall agreement with experiments and coupled-cluster calculations is found for G 0 W 0 results with HSE06 as a starting point including spin-orbit coupling, a truncated Coulomb potential, and an analytical treatment of the singularity at q = 0.

Abstract Automatically detecting and locating remote occlusion small objects from the images of complex traffic environments is a valuable and challenging research. Since the boundary box location is not sufficiently accurate and it is difficult to distinguish overlapping and occluded objects, the authors propose a network model with a second‐order term attention mechanism and occlusion loss. First, the backbone network is built on CSPDarkNet53. Then a method is designed for the feature extraction network based on an item‐wise attention mechanism, which uses the filtered weighted feature vector to replace the original residual fusion and adds a second‐order term to reduce the information loss in the process of fusion and accelerate the convergence of the model. Finally, an objected occlusion regression loss function is studied to reduce the problems of missed detections caused by dense objects. Sufficient experimental results demonstrate that the authors’ method achieved state‐of‐the‐art performance without reducing the detection speed. The mAP@.5 of the method is 85.8% on the Foggy_cityscapes dataset and the mAP@.5 of the method is 97.8% on the KITTI dataset.

Abstract As human‐machine interaction (HMI) in healthcare continues to evolve, the issue of trust in HMI in healthcare has been raised and explored. It is critical for the development and safety of healthcare that humans have proper trust in medical machines. Intelligent machines that have applied machine learning (ML) technologies continue to penetrate deeper into the medical environment, which also places higher demands on intelligent healthcare. In order to make machines play a role in HMI in healthcare more effectively and make human‐machine cooperation more harmonious, the authors need to build good human‐machine trust (HMT) in healthcare. This article provides a systematic overview of the prominent research on ML and HMT in healthcare. In addition, this study explores and analyses ML and three important factors that influence HMT in healthcare, and then proposes a HMT model in healthcare. Finally, general trends are summarised and issues to consider addressing in future research on HMT in healthcare are identified.

Abstract Person image generation aims to generate images that maintain the original human appearance in different target poses. Recent works have revealed that the critical element in achieving this task is the alignment of appearance domain and pose domain. Previous alignment methods, such as appearance flow warping, correspondence learning and cross attention, often encounter challenges when it comes to producing fine texture details. These approaches suffer from limitations in accurately estimating appearance flows due to the lack of global receptive field. Alternatively, they can only perform cross‐domain alignment on high‐level feature maps with small spatial dimensions since the computational complexity increases quadratically with larger feature sizes. In this article, the significance of multi‐scale alignment, in both low‐level and high‐level domains, for ensuring reliable cross‐domain alignment of appearance and pose is demonstrated. To this end, a novel and effective method, named Multi‐scale Cross‐domain Alignment (MCA) is proposed. Firstly, MCA adopts global context aggregation transformer to model multi‐scale interaction between pose and appearance inputs, which employs pair‐wise window‐based cross attention. Furthermore, leveraging the integrated global source information for each target position, MCA applies flexible flow prediction head and point correlation to effectively conduct warping and fusing for final transformed person image generation. Our proposed MCA achieves superior performance on two popular datasets than other methods, which verifies the effectiveness of our approach.

Abstract Eye health has become a global health concern and attracted broad attention. Over the years, researchers have proposed many state‐of‐the‐art convolutional neural networks (CNNs) to assist ophthalmologists in diagnosing ocular diseases efficiently and precisely. However, most existing methods were dedicated to constructing sophisticated CNNs, inevitably ignoring the trade‐off between performance and model complexity. To alleviate this paradox, this paper proposes a lightweight yet efficient network architecture, mixed‐decomposed convolutional network (MDNet), to recognise ocular diseases. In MDNet, we introduce a novel mixed‐decomposed depthwise convolution method, which takes advantage of depthwise convolution and depthwise dilated convolution operations to capture low‐resolution and high‐resolution patterns by using fewer computations and fewer parameters. We conduct extensive experiments on the clinical anterior segment optical coherence tomography (AS‐OCT), LAG, University of California San Diego, and CIFAR‐100 datasets. The results show our MDNet achieves a better trade‐off between the performance and model complexity than efficient CNNs including MobileNets and MixNets. Specifically, our MDNet outperforms MobileNets by 2.5% of accuracy by using 22% fewer parameters and 30% fewer computations on the AS‐OCT dataset.

Abstract Non‐destructive detection of wire bonding defects in integrated circuits (IC) is critical for ensuring product quality after packaging. Image‐processing‐based methods do not provide a detailed evaluation of the three‐dimensional defects of the bonding wire. Therefore, a method of 3D reconstruction and pattern recognition of wire defects based on stereo vision, which can achieve non‐destructive detection of bonding wire defects is proposed. The contour features of bonding wires and other electronic components in the depth image is analysed to complete the 3D reconstruction of the bonding wires. Especially to filter the noisy point cloud and obtain an accurate point cloud of the bonding wire surface, a point cloud segmentation method based on spatial surface feature detection (SFD) was proposed. SFD can extract more distinct features from the bonding wire surface during the point cloud segmentation process. Furthermore, in the defect detection process, a directional discretisation descriptor with multiple local normal vectors is designed for defect pattern recognition of bonding wires. The descriptor combines local and global features of wire and can describe the spatial variation trends and structural features of wires. The experimental results show that the method can complete the 3D reconstruction and defect pattern recognition of bonding wires, and the average accuracy of defect recognition is 96.47%, which meets the production requirements of bonding wire defect detection.

Abstract Network embedding aspires to learn a low‐dimensional vector of each node in networks, which can apply to diverse data mining tasks. In real‐life, many networks include rich attributes and temporal information. However, most existing embedding approaches ignore either temporal information or network attributes. A self‐attention based architecture using higher‐order weights and node attributes for both static and temporal attributed network embedding is presented in this article. A random walk sampling algorithm based on higher‐order weights and node attributes to capture network topological features is presented. For static attributed networks, the algorithm incorporates first‐order to k‐order weights, and node attribute similarities into one weighted graph to preserve topological features of networks. For temporal attribute networks, the algorithm incorporates previous snapshots of networks containing first‐order to k‐order weights, and nodes attribute similarities into one weighted graph. In addition, the algorithm utilises a damping factor to ensure that the more recent snapshots allocate a greater weight. Attribute features are then incorporated into topological features. Next, the authors adopt the most advanced architecture, Self‐Attention Networks, to learn node representations. Experimental results on node classification of static attributed networks and link prediction of temporal attributed networks reveal that our proposed approach is competitive against diverse state‐of‐the‐art baseline approaches.

Abstract Cloud computing has drastically changed the delivery and consumption of live streaming content. The designs, challenges, and possible uses of cloud computing for live streaming are studied. A comprehensive overview of the technical and business issues surrounding cloud‐based live streaming is provided, including the benefits of cloud computing, the various live streaming architectures, and the challenges that live streaming service providers face in delivering high‐quality, real‐time services. The different techniques used to improve the performance of video streaming, such as adaptive bit‐rate streaming, multicast distribution, and edge computing are discussed and the necessity of low‐latency and high‐quality video transmission in cloud‐based live streaming is underlined. Issues such as improving user experience and live streaming service performance using cutting‐edge technology, like artificial intelligence and machine learning are discussed. In addition, the legal and regulatory implications of cloud‐based live streaming, including issues with network neutrality, data privacy, and content moderation are addressed. The future of cloud computing for live streaming is examined in the section that follows, and it looks at the most likely new developments in terms of trends and technology. For technology vendors, live streaming service providers, and regulators, the findings have major policy‐relevant implications. Suggestions on how stakeholders should address these concerns and take advantage of the potential presented by this rapidly evolving sector, as well as insights into the key challenges and opportunities associated with cloud‐based live streaming are provided.

Abstract Policy evaluation (PE) is a critical sub‐problem in reinforcement learning, which estimates the value function for a given policy and can be used for policy improvement. However, there still exist some limitations in current PE methods, such as low sample efficiency and local convergence, especially on complex tasks. In this study, a novel PE algorithm called Least‐Squares Truncated Temporal‐Difference learning (LST2D) is proposed. In LST2D, an adaptive truncation mechanism is designed, which effectively takes advantage of the fast convergence property of Least‐Squares Temporal Difference learning and the asymptotic convergence property of Temporal Difference learning (TD). Then, two feature pre‐training methods are utilised to improve the approximation ability of LST2D. Furthermore, an Actor‐Critic algorithm based on LST2D and pre‐trained feature representations (ACLPF) is proposed, where LST2D is integrated into the critic network to improve learning‐prediction efficiency. Comprehensive simulation studies were conducted on four robotic tasks, and the corresponding results illustrate the effectiveness of LST2D. The proposed ACLPF algorithm outperformed DQN, ACER and PPO in terms of sample efficiency and stability, which demonstrated that LST2D can be applied to online learning control problems by incorporating it into the actor‐critic architecture.

Abstract Few‐shot image classification is the task of classifying novel classes using extremely limited labelled samples. To perform classification using the limited samples, one solution is to learn the feature alignment (FA) information between the labelled and unlabelled sample features. Most FA methods use the feature mean as the class prototype and calculate the correlation between prototype and unlabelled features to learn an alignment strategy. However, mean prototypes tend to degenerate informative features because spatial features at the same position may not be equally important for the final classification, leading to inaccurate correlation calculations. Therefore, the authors propose an effective intraclass FA strategy that aggregates semantically similar spatial features from an adaptive reference prototype in low‐dimensional feature space to obtain an informative prototype feature map for precise correlation computation. Moreover, a dual correlation module to learn the hard and soft correlations was developed by the authors. This module combines the correlation information between the prototype and unlabelled features in both the original and learnable feature spaces, aiming to produce a comprehensive cross‐correlation between the prototypes and unlabelled features. Using both FA and cross‐attention modules, our model can maintain informative class features and capture important shared features for classification. Experimental results on three few‐shot classification benchmarks show that the proposed method outperformed related methods and resulted in a 3% performance boost in the 1‐shot setting by inserting the proposed module into the related methods.

Abstract Three‐way concept analysis is an important tool for information processing, and rule acquisition is one of the research hotspots of three‐way concept analysis. However, compared with three‐way concept lattices, three‐way semi‐concept lattices have three‐way operators with weaker constraints, which can generate more concepts. In this article, the problem of rule acquisition for three‐way semi‐concept lattices is discussed in general. The authors construct the finer relation of three‐way semi‐concept lattices, and propose a method of rule acquisition for three‐way semi‐concept lattices. The authors also discuss the set of decision rules and the relationships of decision rules among object‐induced three‐way semi‐concept lattices, object‐induced three‐way concept lattices, classical concept lattices and semi‐concept lattices. Finally, examples are provided to illustrate the validity of our conclusions.

Abstract In traditional secret image sharing schemes, a secret image is shared among shareholders who have the same position. But if the shareholders have two different positions, essential and non‐essential, it is necessary to use essential secret image sharing schemes. In this article, a verifiable essential secret image sharing scheme based on HLRs is proposed. Shareholder's share consists of two parts. The first part is produced by the shareholders, which prevents the fraud of dealers. The second part is a shadow image that is produced by using HLRs and the first part of share. The verification of the first part of the shares is done for the first time by using multilinear and bilinear maps. Also, for verifying shadow images, Bloom Filters are used for the first time. The proposed scheme is more efficient than similar schemes, and for the first part of the shares, has formal security.

Abstract Disturbance observer‐based control method has achieved good results in the car‐following scenario of intelligent and connected vehicle (ICV). However, the gain of conventional extended disturbance observer (EDO)‐based control method is usually set manually rather than adjusted adaptively according to real time traffic conditions, thus declining the car‐following performance. To solve this problem, a car‐following strategy of ICV using EDO adjusted by reinforcement learning is proposed. Different from the conventional method, the gain of proposed strategy can be adjusted by reinforcement learning to improve its estimation accuracy. Since the “equivalent disturbance” can be compensated by EDO to a great extent, the disturbance rejection ability of the car‐following method will be improved significantly. Both Lyapunov approach and numerical simulations are carried out to verify the effectiveness of the proposed method.

Abstract Rough set theory places great importance on approximation accuracy, which is used to gauge how well a rough set model describes a target concept. However, traditional approximation accuracy has limitations since it varies with changes in the target concept and cannot evaluate the overall descriptive ability of a rough set model. To overcome this, two types of average approximation accuracy that objectively assess a rough set model’s ability to approximate all information granules is proposed. The first is the relative average approximation accuracy, which is based on all sets in the universe and has several basic properties. The second is the absolute average approximation accuracy, which is based on undefinable sets and has yielded significant conclusions. We also explore the relationship between these two types of average approximation accuracy. Finally, the average approximation accuracy has practical applications in addressing missing attribute values in incomplete information tables.

Abstract Facial beauty analysis is an important topic in human society. It may be used as a guidance for face beautification applications such as cosmetic surgery. Deep neural networks (DNNs) have recently been adopted for facial beauty analysis and have achieved remarkable performance. However, most existing DNN‐based models regard facial beauty analysis as a normal classification task. They ignore important prior knowledge in traditional machine learning models which illustrate the significant contribution of the geometric features in facial beauty analysis. To be specific, landmarks of the whole face and facial organs are introduced to extract geometric features to make the decision. Inspired by this, we introduce a novel dual‐branch network for facial beauty analysis: one branch takes the Swin Transformer as the backbone to model the full face and global patterns, and another branch focuses on the masked facial organs with the residual network to model the local patterns of certain facial parts. Additionally, the designed multi‐scale feature fusion module can further facilitate our network to learn complementary semantic information between the two branches. In model optimisation, we propose a hybrid loss function, where especially geometric regulation is introduced by regressing the facial landmarks and it can force the extracted features to convey facial geometric features. Experiments performed on the SCUT‐FBP5500 dataset and the SCUT‐FBP dataset demonstrate that our model outperforms the state‐of‐the‐art convolutional neural networks models, which proves the effectiveness of the proposed geometric regularisation and dual‐branch structure with the hybrid network. To the best of our knowledge, this is the first study to introduce a Vision Transformer into the facial beauty analysis task.

Abstract Redundancy elimination techniques are extensively investigated to reduce storage overheads for cloud‐assisted health systems. Deduplication eliminates the redundancy of duplicate blocks by storing one physical instance referenced by multiple duplicates. Delta compression is usually regarded as a complementary technique to deduplication to further remove the redundancy of similar blocks, but our observations indicate that this is disobedient when data have sparse duplicate blocks. In addition, there are many overlapped deltas in the resemblance detection process of post‐deduplication delta compression, which hinders the efficiency of delta compression and the index phase of resemblance detection inquires abundant non‐similar blocks, resulting in inefficient system throughput. Therefore, a multi‐feature‐based redundancy elimination scheme, called MFRE, is proposed to solve these problems. The similarity feature and temporal locality feature are excavated to assist redundancy elimination where the similarity feature well expresses the duplicate attribute. Then, similarity‐based dynamic post‐deduplication delta compression and temporal locality‐based dynamic delta compression discover more similar base blocks to minimise overlapped deltas and improve compression ratios. Moreover, the clustering method based on block‐relationship and the feature index strategy based on bloom filters reduce IO overheads and improve system throughput. Experiments demonstrate that the proposed method, compared to the state‐of‐the‐art method, improves the compression ratio and system throughput by 9.68% and 50%, respectively.

Abstract Robot calligraphy visually reflects the motion capability of robotic manipulators. While traditional researches mainly focus on image generation and the writing of simple calligraphic strokes or characters, this article presents a generative adversarial network (GAN)‐based motion learning method for robotic calligraphy synthesis (Gan2CS) that can enhance the efficiency in writing complex calligraphy words and reproducing classic calligraphy works. The key technologies in the proposed approach include: (1) adopting the GAN to learn the motion parameters from the robot writing operation; (2) converting the learnt motion data into the style font and realising the transition from static calligraphy images to dynamic writing demonstration; (3) reproducing high‐precision calligraphy works by synthesising the writing motion data hierarchically. In this study, the motion trajectories of sample calligraphy images are firstly extracted and converted into the robot module. The robot performs the writing with motion planning, and the writing motion parameters of calligraphy strokes are learnt with GANs. Then the motion data of basic strokes is synthesised based on the hierarchical process of ‘stroke‐radical‐part‐character’. And the robot re‐writes the synthesised characters whose similarity with the original calligraphy characters is evaluated. Regular calligraphy characters have been tested in the experiments for method validation and the results validated that the robot can actualise the robotic calligraphy synthesis of writing motion data with GAN.

Abstract In recent years, intelligent robots are extensively applied in the field of the industry and intelligent rehabilitation, wherein the human‐robot interaction (HRI) control strategy is a momentous part that needs to be ameliorated. Specially, the efficacy and robustness of the HRI control algorithm in the presence of unknown external disturbances deserve to be addressed. To deal with these urgent issues, in this study, artificial systems, computational experiments and a parallel execution intelligent control framework are constructed for the HRI control. The upper limb‐robotic exoskeleton system is re‐modelled as an artificial system. Depending on surface electromyogram‐based subject's active motion intention in the practical system, a non‐convex function activated anti‐disturbance zeroing neurodynamic (NC‐ADZND) controller is devised in the artificial system for parallel interaction and HRI control with the practical system. Furthermore, the linear activation function‐based zeroing neurodynamic (LAF‐ZND) controller and proportional‐derivative (posterior deltoid (PD)) controller are presented and compared. Theoretical results substantiate the global convergence and robustness of the proposed controller in the presence of different external disturbances. In addition, the simulation results verify that the NC‐ADZND controller is better than the LAF‐ZND and the PD controllers in respect of convergence order and anti‐disturbance characteristics.

Abstract Humanoid robots have attracted much attention by virtue of their compatibility with human environments. However, biped humanoids with immense promise still cannot function steadily and reliably in real‐world settings in the current state. Hence, rationally combining a humanoid robot with different stable mobile platforms is a favoured solution for diverse scenarios. Here, a new versatile humanoid robot platform, aiming to provide a generic solution that can be flexibly deployed in diverse scenarios, for example, indoors and fields is presented. Versatile humanoid robot platform incorporates multimodal perception, and extensible interfaces on hardware and software, allowing it to be rapidly integrated with different mobile platforms and end‐effectors, only through easy‐to‐assemble interfaces. Additionally, the platform has achieved impressive integration, lightness, dexterity, and strength in its class, with human‐like size and rich perception, targeted to have human‐intelligent manipulation skills for human‐engineered environments. Overall, this article elaborates on the reasoning behind the design choices, and outlines each subsystem. Lastly, the essential performance of the platform is successfully demonstrated in a set of experiments with precise and dexterous manipulation, and human–robot collaboration requirements.

Abstract In industrial manufacturing, how to accurately classify defective products and locate the location of defects has always been a concern. Previous studies mainly measured similarity based on extracting single‐scale features of samples. However, only using the features of a single scale is hard to represent different sizes and types of anomalies. Therefore, the authors propose a set of memory banks of multi‐scale features (MBMF) to enrich feature representation and detect and locate various anomalies. To extract features of different scales, different aggregation functions are designed to produce the feature maps at different granularity. Based on the multi‐scale features of normal samples, the MBMF are constructed. Meanwhile, to better adapt to the feature distribution of the training samples, the authors proposed a new iterative updating method for the memory banks. Testing on the widely used and challenging dataset of MVTec AD, the proposed MBMF achieves competitive image‐level anomaly detection performance (Image‐level Area Under the Receiver Operator Curve (AUROC)) and pixel‐level anomaly segmentation performance (Pixel‐level AUROC). To further evaluate the generalisation of the proposed method, we also implement anomaly detection on the BeanTech AD dataset, a commonly used dataset in the field of anomaly detection, and the Fashion‐MNIST dataset, a widely used dataset in the field of image classification. The experimental results also verify the effectiveness of the proposed method.